Automated Classification of Storm Flashing / Non-flashing Condition from Microphysical and Environmental Observations

Three years of paired TRMM radar, passive microwave (PM), lightning and NCEP reanalysis data are used to answer the question: “Can we classify a storm as flashing, or not, based only on microphysical and environmental observations?”. Univariate, rule-based threshold models are tested, as well as multivariate linear (discriminant analysis) and multivariate nonlinear (neural network) logistic regressions. Simple reflectivity threshold rules achieve, over tropical land, 75% unbiased true skill (HSS), 79% probability of detection (POD) and 25% false alarm ratio (FAT), although their performance varies geographically. Multivariate neural network models achieve, over tropical land, 80% HSS, 82% POD and 19% FAT, exhibit much less geographic bias, are more robust over a range of decision thresholds, and are not overfitted. Automated classification from PM observations (DMSP/SSM-I, TRMM/TMI, Aqua/AMSR-E, NPOESS/ CMIS) is feasible and opens the possibility of a 20-yr “backfilled” global lightning climatology. INTRODUCTION: Recent advances in ground-based and orbital lightning detection now provide some form of direct lightning observation over most of the globe [Christian et al, 2003]. However, these observations are often highly undersampled (e.g., 0.3% of all space-time from NASA’s Low-Earth-Orbiting optical detectors) or taken with very low detection efficiency (ground network observations at distant range). Needs (e.g., aviation) still exist for robust and automated classification of lightning hazard in remote regions, in absence of direct observations. Field programs have suggested that storm microphysical properties, e.g., radar reflectivity in the mixed phase region, might serve as proxies for lightning occurrence [e.g., Larsen and Stansbury, 1974; Petersen et al, 1996]. The Tropical Rainfall Measuring Mission (TRMM) satellite simultaneously measures radar reflectivity, PM brightness temperature and lightning flash rate, and provides a very large sample training/validation database for algorithm development, sufficient to both avoid model overfitting and generalize well beyond the local conditions of field programs. Development of any classification model essentially involves specification of a function for the “decision surface”, in an n-parameter input space, between members of various categories; in this case, flashing and nonflashing storms. The simplest model is a scalar threshold placed on a single input (a univariate rule-based model). Univariate or multivariate rule-based models have the benefit of being easily understandable, but are limited in their ability to robustly parameterize the decision surface. Multivariate regression is a reasonable way to parameterize this surface, and indeed is the basis for traditional discriminant analysis (a multivariate, MSE, linear regression of the input parameters to define the surface). Classification neural networks are an example of multivariate, logistic nonlinear regressions, which allow both interactions between, and nonlinear transformations of, input variates, without a priori specification of model functional form. In this study the simplest (rule based) and most sophisticated (neural network) decision surface parameterizations are compared. METHODOLOGY: Three years (12/97–11/00) of TRMM radar and PM observations from 35S-35N are used as input data, while TRMM lightning data are used as model truth. NCEP reanalysis [Kalnay et al, 1996] environmental variates (both surface and tropospheric) are also used as inputs; we allow the TRMM satellite to “virtually” sample the NCEP 4-D atmosphere and tag each storm observation with climatological (20-yr monthly mean) and concurrent (daily anomaly) environmental data. Storms are defined using the University of Utah “precipitation feature” approach [Nesbitt et al, 2000]; loosely, a storm is defined as contiguous raining TRMM radar pixel columns with adjacent anvil overhang, if any. Storms are only considered during “warm” seasons; the NCEP surface temperature at the storm time/location must be greater than 15C (arbitrarily chosen). Additionally, the 0C level must lie between 3-6 km altitude (thus excluding high mountain storms), and only cold-topped (15 dBZ tops < 0C) storms are considered. A total of 519,425 land storms and 1,004,003 ocean storms were considered. A storm is considered “flashing” if IC or CG lightning is detected within 10 km of any of its radar columns. Given the TRMM/LIS detection efficiency, “flashing” storms are thus those with flash rates > ~1 fl/min [Boccippio et al, 2002]. Radar-based inputs include the temperature of (15, 20, ... 40 dBZ) echo tops (with nonoccurrence set to the surface temperature), maximum reflectivity at (0, -5, ... -40C) expressed as Ze^0.5, total area of the storm colder than –10C and greater than (20, 25 ... 40 dBZ), total volume of the storm meeting the same criteria, and total ice mass in the storm, using a simple Z-M relationship [Petersen and Rutledge, 2001]. The percentages of the storm classified by the TRMM 2A23 algorithm as convective, stratiform and non-raining were also input. PM-based inputs include the minimum 85 and 37 GHz brightness temperatures, the total storm area at 85 GHz colder than 275, 250, 225 and 200K, and at 37 GHz colder than 280, 270 and 260K. NCEP inputs include surface temperature, pressure, sensible and latent heat fluxes, and Bowen ratio, lower troposphere lifted index and lapse rate (850-500 mb), column precipitable water, relative humidity at the surface, 850, 500 and 300 mb, vertical pressure velocity at 850, 500 and 300 mb, and tropopause pressure. Radar-based threshold rule models were tested for the maximum reflectivity at each temperature level; the model consisting of the temperature level (input) and reflectivity decision threshold yielding the highest unbiased true skill was selected as the “best” univariate model. Neural networks including radar-only, radar+NCEP, PMonly, PM+NCEP, and radar+PM+NCEP inputs were trained. The latter (most sophisticated) network included 87 inputs and 15 hidden nodes, and used a cross-entropy rather than MSE error minimization criterion (appropriate for dichotomous flashing/non-flashing outputs [Marzban and Witt, 2001]). Network structure and training hyperparameters were not optimized; the networks shown are simply reasonably well-performing models, not the best possible. Overfitting was prevented by splitting the storm cases into 2/3 and 1/3 subsets. The 2/3 partition was used for training and validation; it was repeatedly split in half, with half used for model parameter estimation and half used to assess the onset of overfitting. After 25 such splits, an optimal convergence criterion was determined and the entire 2/3 partition used for parameter estimation. The remaining 1/3 partition was reserved and used to test the fitted models; <1% differences in skill scores were found between the training/validation and testing data, confirming that the nonlinear models were not overfitted. Performance was assessed using conventional contingency-table based statistics [Marzban, 1998]: the Heidke Skill Score (loosely, “true” skill after correcting for correct diagnoses due to chance), the Gilbert Score (loosely, “threat” skill in real or forecast lightning situations, after correcting for correct diagnoses due to chance), the Probability of Detection (POD) and the False Alarm Ratio (FAT). Skill is also diagnosed in a non-scalar fashion by constructing Receiver Operating Characteristic (ROC) plots; i.e., parametric plots of POD vs FAT as all possible decision thresholds on the model outputs are considered. ROC plots help show model robustness (some end-users, e.g., may be constrained to use a model which limits the maximum allowable FAT, rather than maximizing HSS). The total area under ROC curves provides an additional scalar measure of model skill and robustness. RESULTS: The formally “optimal” maximum-reflectivity based univariate threshold rule (yielding the highest HSS) is found to be: maximum dBZ @ -20C > 29 dBZ. This yields unbiased true skill (HSS) of 0.80, unbiased threat skill (GS) of 0.60, POD of 0.79 and FAT of 0.25, over land within the experiment domain. Practically, however, maximum reflectivity at any level from –14 to –21C, with thresholds from 33 dBZ to 28 dBZ over this range, yield virtually the same skill scores. This is remarkably similar to the “rules-ofthumb” gleaned from individual field programs (30-35 dBZ occurrence in the mixed phase region) which typically consider much smaller sample sizes, although optimal performance is strongly dependent upon isolating the best input and threshold in the mixed phase region. Fig. 1 shows the skill score performance as thresholds are varied. Over oceans, the “optimal” rule is: maximum dBZ @ -15C > 35.5 dBZ (effectively a “stronger” criterion than at the equivalent temperature level over land); this yields HSS/GS/POD/FAT of 0.59 / 0.42 / 0.59 / 0.40, significantly lower skill. Training of a neural network including all radar and NCEP inputs improved skill scores, over land, to 0.80 / 0.66 / 0.80 / 0.18, effectively lowering the false alarm incidence by 7% (and hence improving the threat skill) without compromising the probability of detection. Over ocean, this approach Fig 1:Skill scores for the “best” (highest true skill) land [upper] and ocean [lower] maximum-reflectivty rule-based models. HSS